{
    "cells": [
        {
            "cell_type": "markdown",
            "id": "307804a3-c02b-4a57-ac0d-172c30ddc851",
            "metadata": {},
            "source": [
                "# Qdrant Index Demo"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "f7010b1d-d1bb-4f08-9309-a328bb4ea396",
            "metadata": {},
            "source": [
                "#### Creating a Qdrant client"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 1,
            "id": "47264e32",
            "metadata": {
                "ExecuteTime": {
                    "end_time": "2023-02-10T12:20:23.988789Z",
                    "start_time": "2023-02-10T12:20:23.967877Z"
                }
            },
            "outputs": [],
            "source": [
                "import logging\n",
                "import sys\n",
                "\n",
                "logging.basicConfig(stream=sys.stdout, level=logging.INFO)\n",
                "logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 2,
            "id": "0c9f4d21-145a-401e-95ff-ccb259e8ef84",
            "metadata": {
                "ExecuteTime": {
                    "end_time": "2023-02-10T12:20:24.908956Z",
                    "start_time": "2023-02-10T12:20:24.537064Z"
                },
                "pycharm": {
                    "is_executing": true
                }
            },
            "outputs": [],
            "source": [
                "import qdrant_client"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 3,
            "id": "b1757e8a",
            "metadata": {
                "ExecuteTime": {
                    "end_time": "2023-02-10T12:20:24.986059Z",
                    "start_time": "2023-02-10T12:20:24.910787Z"
                },
                "pycharm": {
                    "is_executing": true
                }
            },
            "outputs": [],
            "source": [
                "client = qdrant_client.QdrantClient(\n",
                "    host=\"<qdrant-host>\",\n",
                "    api_key=\"<qdrant-api-key>\", # For Qdrant Cloud, None for local instance\n",
                "    https=True, # True if using Qdrant Cloud\n",
                ")"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "8ee4473a-094f-4d0a-a825-e1213db07240",
            "metadata": {},
            "source": [
                "#### Load documents, build the GPTQdrantIndex"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 4,
            "id": "0a2bcc07",
            "metadata": {
                "ExecuteTime": {
                    "end_time": "2023-02-10T12:20:30.170536Z",
                    "start_time": "2023-02-10T12:20:29.402794Z"
                },
                "pycharm": {
                    "is_executing": true
                }
            },
            "outputs": [],
            "source": [
                "from gpt_index import GPTQdrantIndex, SimpleDirectoryReader\n",
                "from IPython.display import Markdown, display"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 5,
            "id": "68cbd239-880e-41a3-98d8-dbb3fab55431",
            "metadata": {
                "ExecuteTime": {
                    "end_time": "2023-02-10T12:20:30.175678Z",
                    "start_time": "2023-02-10T12:20:30.172456Z"
                },
                "pycharm": {
                    "is_executing": true
                }
            },
            "outputs": [],
            "source": [
                "# load documents\n",
                "documents = SimpleDirectoryReader('../paul_graham_essay/data').load_data()"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 6,
            "id": "ba1558b3",
            "metadata": {
                "ExecuteTime": {
                    "end_time": "2023-02-10T12:20:33.735897Z",
                    "start_time": "2023-02-10T12:20:30.404245Z"
                },
                "pycharm": {
                    "is_executing": true
                }
            },
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "INFO:root:> [build_index_from_documents] Total LLM token usage: 0 tokens\n",
                        "> [build_index_from_documents] Total LLM token usage: 0 tokens\n",
                        "INFO:root:> [build_index_from_documents] Total embedding token usage: 17621 tokens\n",
                        "> [build_index_from_documents] Total embedding token usage: 17621 tokens\n"
                    ]
                }
            ],
            "source": [
                "index = GPTQdrantIndex.from_documents(documents, client=client, collection_name=\"paul_graham\")"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 7,
            "id": "c36cadc1",
            "metadata": {
                "ExecuteTime": {
                    "end_time": "2023-02-10T12:20:33.742793Z",
                    "start_time": "2023-02-10T12:20:33.738289Z"
                }
            },
            "outputs": [],
            "source": [
                "# save index to disk\n",
                "index.save_to_disk(\"index_qdrant.json\")"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 8,
            "id": "70b372a7",
            "metadata": {
                "ExecuteTime": {
                    "end_time": "2023-02-10T12:20:33.821131Z",
                    "start_time": "2023-02-10T12:20:33.745017Z"
                }
            },
            "outputs": [],
            "source": [
                "# load index from disk\n",
                "index = GPTQdrantIndex.load_from_disk(\"index_qdrant.json\", client=client)"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "04304299-fc3e-40a0-8600-f50c3292767e",
            "metadata": {},
            "source": [
                "#### Query Index"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 9,
            "id": "35369eda",
            "metadata": {
                "ExecuteTime": {
                    "end_time": "2023-02-10T12:20:51.328762Z",
                    "start_time": "2023-02-10T12:20:33.822688Z"
                }
            },
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "INFO:root:> [query] Total LLM token usage: 3841 tokens\n",
                        "> [query] Total LLM token usage: 3841 tokens\n",
                        "INFO:root:> [query] Total embedding token usage: 8 tokens\n",
                        "> [query] Total embedding token usage: 8 tokens\n"
                    ]
                }
            ],
            "source": [
                "# set Logging to DEBUG for more detailed outputs\n",
                "response = index.query(\"What did the author do growing up?\")"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 10,
            "id": "bedbb693-725f-478f-be26-fa7180ea38b2",
            "metadata": {
                "ExecuteTime": {
                    "end_time": "2023-02-10T12:20:51.337062Z",
                    "start_time": "2023-02-10T12:20:51.330857Z"
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/markdown": [
                            "<b>\n",
                            "\n",
                            "The author grew up writing short stories, programming on an IBM 1401, and building a computer kit with a friend. He also wrote simple games, a program to predict how high his model rockets would fly, and a word processor. He studied philosophy in college, but switched to AI and taught himself Lisp. He wrote a book about Lisp hacking and reverse-engineered SHRDLU. He also took art classes at Harvard and applied to art schools, but was disappointed by the lack of teaching and learning in the painting department at the Accademia. He also had experience with 19th century studio painting conventions, such as having a little stove fed with kindling and a nude model sitting as close to it as possible.</b>"
                        ],
                        "text/plain": [
                            "<IPython.core.display.Markdown object>"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                }
            ],
            "source": [
                "display(Markdown(f\"<b>{response}</b>\"))"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 11,
            "id": "99212d33",
            "metadata": {
                "ExecuteTime": {
                    "end_time": "2023-02-10T12:21:10.337294Z",
                    "start_time": "2023-02-10T12:20:51.338718Z"
                }
            },
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "INFO:root:> [query] Total LLM token usage: 3899 tokens\n",
                        "> [query] Total LLM token usage: 3899 tokens\n",
                        "INFO:root:> [query] Total embedding token usage: 14 tokens\n",
                        "> [query] Total embedding token usage: 14 tokens\n"
                    ]
                }
            ],
            "source": [
                "# set Logging to DEBUG for more detailed outputs\n",
                "response = index.query(\"What did the author do after his time at Y Combinator?\")"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 12,
            "id": "1a720ad6",
            "metadata": {
                "ExecuteTime": {
                    "end_time": "2023-02-10T12:21:10.355872Z",
                    "start_time": "2023-02-10T12:21:10.343486Z"
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/markdown": [
                            "<b>\n",
                            "\n",
                            "After his time at Y Combinator, the author wrote essays, worked on Lisp, painted, and moved to England. He wrote a new Lisp called Bel, which took four years to complete. After finishing Bel, he wrote a bunch of essays and started to think about other things he could work on. He wrote an essay for himself to answer the question of how to choose what to work on, and then wrote a more detailed version for others to read. He also reflected on the exponential growth of commodity processors and the tight coupling of money and coolness in the art world. He also found a rent-stabilized apartment in New York, which was much cheaper than market price. Finally, he worked on software that could be launched as soon as it was done.</b>"
                        ],
                        "text/plain": [
                            "<IPython.core.display.Markdown object>"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                }
            ],
            "source": [
                "display(Markdown(f\"<b>{response}</b>\"))"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "id": "b0b6d770",
            "metadata": {},
            "outputs": [],
            "source": []
        }
    ],
    "metadata": {
        "kernelspec": {
            "display_name": "Python 3 (ipykernel)",
            "language": "python",
            "name": "python3"
        },
        "language_info": {
            "codemirror_mode": {
                "name": "ipython",
                "version": 3
            },
            "file_extension": ".py",
            "mimetype": "text/x-python",
            "name": "python",
            "nbconvert_exporter": "python",
            "pygments_lexer": "ipython3",
            "version": "3.9.16"
        }
    },
    "nbformat": 4,
    "nbformat_minor": 5
}